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In  manned  systems,  performance  can  change  significantly 
with  changes  in  display  design.  With  today's  computer  and  display 
technology,  it  is  possible  to  provide  virtually  any  display  function 
desired  including  automating  many  of  the  information  processing  tasks 
previously  performed  by  the  human  operator.  However,  the  relation- 
ship between  display  design  and  total  system  (man  and  machine)  per- 
formance must  be  known  in  order  to  systematically  select  the  display 
features . 


A new  human  operator  modeling  technique  termed 
"operator  measures  and  criteria"  (OMAC)  was  used  to  represent 
the  ship  control  performance  of  the  Officer  of  the  Deck  (OOD). 
OMAC's  are  measures  and  criteria  which  are  determined  by 
calculation  to  be  those  optimized  by  the  observed  OOD  controlled 
ship  responses.  OMAC's  were  determined  from  data  obtained  from 
an  experiment  in  which  three  different  displays  were  used.  With 
each  of  those  displays,  OOD  subjects  demonstrating  superior  per- 
formance resulted  in  identical  criteria,  but  an  apparently  self- 
imposed  constraint  called  purview  (range  from  own  ship  within 
which  contacts  are  processed  by  the  OOD),  was  shown  to  be 
different  for  each  display  type.  Purview  is  shown  to  explain 
differences  in  performance  with  different  displays.  The  proportion 
of  OOD  subjects  demonstrating  superior  performance  is  also  shown 
to  be  a function  of  display  type. 
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1 .0  INTRODUCTION 

The  designer  of  a man-machine  system  typically  performs 
his  design  task  with  knowledge  of  the  system  objectives,  human  factors 
principles,  and  a concept  of  the  display  and  control  requirements . 

However,  he  typically  does  not  know  how  effectively  the  human  operator 
will  perform  given  various  display  designs,  nor  does  he  have  a model 
for  predicting  the  human  operator  performance.  If  the  designer  had  a 
model  which  could  predict  significant  performance  differences  with 
different  display  designs,  the  display  features  could  be  selected  in  a 
systematic,  performance  oriented  manner. 

Performance  prediction  models  have  long  been  sought  to 
aid  in  system  design  and  to  better  understand  how  display  designs 
affect  performance.  Present  modeling  methods  (McRuer,  Preyss, 

Sheridan,  Kleinman)  which  typically  represent  the  human  operator's 
moment- to- moment  control  response,  have  not  provided  the  necessary 
predictive  models.  The  modeling  method  used  in  the  analysis  reported 
here  employs  a construct  based  on  optimal  control  theory  to  provide 
performance  prediction. 

Optimal  control  theory  provides  a means  for  determining 
the  system  response  which,  within  the  constraints  imposed,  is  best 
according  to  the  specified  criteria.  Given  criteria  and  constraints, 
optimal  control  theory  permits  evaluation  of  all  possible  responses  that 
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satisfy  the  constraints,  and  in  doing  so  identifies  the  response  that  is 
best  according  to  the  criteria.  Its  property  of  interest  here  is  that  it 
relates  system  responses  to  criteria  and  vice  versa.  With  this  capability, 
it  is  possible  to  model  a response  by  identifying  the  criteria  optimized. 
This  modeling  process  is  the  inverse  of  the  process  used  by  the  designer 
of  an  automatic  control  system  wherein  criteria  are  selected  first  followed  by- 
determination  of  system  responses  that  optimize  that  criteria.  The 


inverse  process  used  in  this  study  starts  with  observed  responses  and 
seeks  to  identify  the  criteria  optimized  by  these  responses.  Rationale 
for  such  a modeling  strategy  is  that  responses  are  situation  specific 
and  thus  do  not  permit  a compact,  general  representation  of  human 
operator  performance.  In  contrast,  criteria  optimized  by  responses 
may  not  be  as  situation  specific  and  thus  may  provide  a model  which 


is  valid  in  many  control  task  situations. 

Several  example  problems  are  presented  in  the  following 
paragraphsto  illustrate  the  modeling  concept  and  especially  to  provide 
operational  descriptions  of  the  terms:  criteria,  constraints,  performance 

measure,  and  performance. 

Consider  first  an  automobile  control  problem  starts  with 
a car  stopped  at  a red  stop  light.  When  the  light  turns  green,  the 
driver  accelerates  the  car  towards  the  next  stop  light  which  is  red. 

The  car  is  brought  to  a nalt  at  that  stop  light.  Three  different 
automobile  responses  are  defined  to  illustrate  the  concepts  involved: 
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Case  I 


A Corvette  is  accelerated  with  maximum  acceleration 
from  the  first  stop  light  to  a point  before  the  second  stop  light 
where  the  break  is  maximally  applied.  The  car  is  brought  to  a 
halt  at  the  second  stop  light.  Travel  time  from  one  stop  light  to 
the  other  is  15  seconds. 

Case  II 

This  case  is  identical  to  Case  I except  the  car  is  a 
V W and  due  to  the  reduced  acceleration  capability,  the  travel  time 
is  25  seconds. 

Case  III 

A Corvette  is  accelerated  at  a constant  but  less  than 
maximum  acceleration  away  from  the  first  light  and  is  breaked  at 
less  than  full  breaking  at  the  second  light.  Travel  time  is  25  seconds. 

The  auto  responses  in  Cases  I and  II  can  be  represented 
by  the  criteria  optimized:  travel  time.  Those  responses  minimize  the  amount 

of  time  required  to  travel  from  one  light  to  the  next.  Thus,  the 
complex  responses  involving  calculation  of  when  to  "hit  the  breaks" 
can  be  modeled  by  simply  stating  that  they  are  time  optimal. 

Transit  time  is  different  in  Cases  I and  II,  nowever,  due 
to  the  differences  in  acceleration  capability.  The  maximum  acceleration 


capability  of  a car  is  constrained,  i.e.,  the  driver  can  choose  any 
acceleration  up  to  the  maximum  but  no  greater.  Thus  the  acceleration 
level  available  is  constrained.  Similarly,  the  amount  of  breaking  force 
available  is  constrained.  While  the  criterion  (transit  time)  optimized 
in  Cases  l and  II  is  the  same,  performance  (measured  by  transit  time) 
is  different  due  to  different  constraints  (acceleration  and  breaking 
capability). 

Note  that  performance  can  be  measured  with  any  measure 
selected.  The  performance  measure  employed  does  not  have  to  be 
the  criterion  optimized.  The  performance  measure  in  the  above 
example  could  have  been  selected  as  fuel  used,  tire  wear,  or  a 
combination  of  these  factors. 

Case  III  provides  a car  response  that  optimizes  the  sum 
of  fuel  and  transit  time  used.  Note  that  the  criterion  optimized 
by  the  Case  III  response  is  different  from  that  of  Cases  I and  II, 
but  performance  as  indicated  by  the  transit  time  performance  measure 
is  the  same  as  for  Case  II. 

Weighting  factors  on  fuel  and  transit  time  such  as: 

Criterion  = A Time  + (1-A)  fuel,  0 < A < 1 

would  permit  modeling  of  many  car  responses  by  simply  finding  the 
value  of  A that  provides  the  criterion  optimized  by  a given  car  response. 
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Consider  now  a more  practical  example.  Consider  a 
ship  control  problem  whose  Officer  of  the  Deck  (OOD)  is  to  direct 
the  ship  from  a starting  location  to  an  objective  location  and  arrive 
at  the  objective  in  90  minutes.  The  ship  is  to  be  controlled  so  as  to 
avoid  all  contacts  (other  ships)  by  at  least  two  miles.  A performance 
measure  for  this  problem  can  be  developed  using  measures  of  transit 
time  and  contact  avoidance.  This  measure  would  penalize  for  excessive 
transit  time  and  for  passing  contacts  closer  than  two  miles.  Ship 
control  also  involves  constraints  such  as  an  upper  limit  to  speed  and  a 
lower  limit  to  turning  radius.  Using  an  analysis  similar  to  that 
described  for  car  control,  a ship  transiting  response  which  may  include 
any  contact  avoidance  maneuvering,  can  be  modeled  by  the  criteria 
optimized.  This  modeling  concept  and  manual  control  problem  are 
the  problems  treated  in  the  analysis  reported  here. 

The  expectation  in  identifying  criteria  optimized  by  responses 
is  that  the  criteria  can  be  used  to: 

1 . Predict  responses  (and  thereby  performance)  in 

many  control  task  situations  - not  just  those  observed 
in  experiments  from  which  the  criteria  are  identified. 

2.  Reveal  the  control  strategies  used  by  human  operators. 

3.  Reveal  variations  in  control  strategy  (and  performance) 
that  may  result  from  variations  in  display  design. 
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The  term  "operator  measures  and  criteria"  (OMAC)  is 
used  to  denote  the  criteria  optimized  by  an  observed  response.  The 
term  "apparent  OMAC"  is  used  to  refer  to  the  apparent  criteria  being 
used  by  the  human  operator.  It  is  not  known  if  he  is  in  fact  using  that 
criteria,  but  it  predicts  his  control  actions  and  resultant  system  responses. 

Returning  to  the  display  design  problem  identified  at  the 
beginning  of  this  section,  previous  research  (Connelly  1976) 
resulted  in;  development  of  methods  of: 

a.  Determining  the  apparent  OMAC, 

b.  Predicting  performance  using  OMAC  shown  to  be 
representative  of  human  operator  performance,  and 

c.  Analyzing  OMAC's  which  are  associated  with 
different  display  types  to  identify  representative 
human  operator  control  strategies. 

Results  of  that  work  showed  that  significant  differences  in  performance 
cars  result  using  different  displays.  Further,  it  was  demonstrated  that 
these  differences  in  performance  may  not  be  revealed  by  summary 
measures  which  measure  the  number  of  ship  collisions  or  near  collisions. 
Instead,  performance  differences  are  revealed  by  transition  measures 
which  detect  ship  responses  leading  to  collisions  or  near  collisions. 

The  research  further  indicated  that,  in  order  to  accurately 
model  observed  subject  performances , the  representative  OMAC's  must 
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include  a circular  boundary,  termed  purview,  for  the  radar  contacts 
considered  relevant  to  the  control  of  one's  own  ship.  Purview  is  not 
a limitation  imposed  by  the  display  but  is  included  in  OMAC  to  represent 
a boundary  apparently  self-imposed  by  the  subject  Officer  of  the  Deck 
(OOD).  Finally,  a limited  analysis  of  OMAC  parameters,  which  include 
purview,  showed  that  purview  is  different  for  different  displays. 

1 . 1 Research  Objectives 

Research  reported  here  is  an  extension  of  the  previous 
work  described  above.  Specifically,  the  objectives  of  this  program 
were  to: 

1 . Validate  a contact  avoidance  performance  measure 
which  predicts  the  probability  of  collisions  or  near 
collisions  as  a function  of  transition  probabilities 
representing  the  relative  motion  of  own  ship  and 
contacts  along  a sequence  of  states  leading  to 
collisions. 

2.  Complete  the  search  of  OMAC  parameters  to  find 
those  providing  the  best  representation  of  each 
subject  performance . 

3.  Identify  OMAC  parameters  that  represent  performance 
with  each  display  type. 
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4.  Analyze  display  content  to  determine  relationships 
among  display  content,  OMAC  parameters,  and 
contact  avoidance  performance. 

1 .2  Summary  of  Results 

Transition  probabilities  representing  the  relative  motion 
of  own  ship,  and  contacts  along  the  sequence  of  states  leading  to 
collisions,  effectively  characterize  contact  avoidance  maneuvering 
and  provide  a sensitive  measure  of  contact  avoidance  performance. 

OMAC's  can  be  identified  to  represent  subject  performances, 
and  to  represent  average  performance  of  a group  of  subjects  performing 
with  each  display  design.  Analysis  of  the  OMAC  parameters  (purview 
and  criteria,  criteria  used  is  a weighted  function  of  contact  avoidance 
performance  and  transit  time)  representing  performance  with  each 
display  design  shows  that  a greater  proportion  of  subjects  using  the 
PACS*  display  provide  superior  performances  than  those  using  the 
OLD*  display.  Also,  superior  performances  with  both  displays  exhibit 
the  same  OMAC  criteria  but  exhibit  a different  purview.  Furthermore, 
performance  differences  with  different  display  types  are  explained  by 
the  respective  difference  in  purview.  Since  a purview  area  can  be 
readily  translated  into  the  number  of  contacts  in  that  area,  a limited 
purview  also  means  a limited  number  of  contacts  are  being  considered 
*PACS  and  OLD  display  configurations  are  defined  in  Section  2.1. 
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by  the  subject  OOD.  Since  this  limit  is  apparently  self-imposed. 


i.e.,  it  is  not  a limit  imposed  by  the  display,  it  may  be  that  the 
factor  limiting  performance  with  each  display  type  is  a limit  to  the 
subjects  information  processing  capability. 

Based  on  the  overall  results  obtained  with  the  OMAC 
method  of  modeling  human  performance,  it  is  concluded  that  this 
criteria  modeling  approach  is  a practical  way  to  model  human  per- 
formance in  an  operational  or  near  operational  problem  environment. 
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2.0  METHOD 

The  investigation  reported  here  used  data  collected  during 
a series  of  experiments  in  which  a subject  acting  as  an  Officer  of 
the  Deck  (OOD)  controlled  a simulated  ship  in  a simulated  environment. 
His  task  was  to  direct  a ship  transit  from  the  initial  point  to  the  terminal 
point,  within  a pre-specified  time  interval,  while  avoiding  simulated 
contacts  along  the  way.  The  experiment,  using  equipment  known  as 
the  Surface  Ship  Bridge  Console  System  (Gawitt/Beary),  was  run  by 
personnel  of  NSRDC,  Annapolis,  Maryland,  for  purposes  other  than 
this  research  program.  The  data  from  that  experiment  was  used  in 
the  research  reported  here. 

The  objectives  of  the  research  were  accomplished  with 
four  analyses.  Analysis  I is  a validation  test  of  a contact  avoidance 
performance  measure,  termed  "transition  measure,"  which  predicts  the 
probability  of  near  collisions  using  transition  probabilities  representing 
the  relative  motion  of  own  ship  and  contacts  along  the  sequence  of  states 
leading  to  collisions.  The  transition  performance  measure  is  used  in 
Analyses  II  and  III. 

OMAC’s  representing  each  subject  performance  trial  are 
determined  by  Analysis  II.  Analysis  III  is  an  analysis  of  the  parameters 
of  OMAC's  representing  subjects  performing  with  each  display  type. 
Representative  OMAC’s  are  used  to  explain  performance  differences 
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obtained  with  different  display  types.  The  PACS  and  OLD  displays 
were  analyzed  in  detail  because  statistically  significant  performances 
were  obtained  with  these  two  displays.  Performance  with  a third 
display  "RVV",  described  in  Section  2.1,  was  analyzed  in  the 
research  previously  reported  and  did  not  result  in  performance 
significantly  different  from  that  with  the  OLD  display. 

Analysis  IV  identifies  the  information  processing  required 
to  perform  the  OOD  ship  control  task  and  identifies  that  portion  of  the 
information  processing  provided  automatically  by  the  PACS  display. 

Prior  to  a detailed  description  of  these  analyses,  a descrip- 
tion of  the  manual  control  task  and  analytical  tools  is  given.  Included 
in  the  analytical  tools  are  methods  of: 

1 . Identifying  an  OMAC  representing  subject  performance, 

2.  Calculating  contact  avoidance  performance  measures, 

3.  Comparing  ship  response  patterns,  and 

4.  Developing  candidate  OMAC's. 

A detailed  description  of  the  display  designs  used  for  the  study  is  given 
in  the  Appendix. 

2 . 1 Description  of  the  Manual  Control  Task 

The  subject,  acting  as  an  Officer  of  the  Deck  (OOD),  commands 
the  ship  course  and  speed  required  to  transit  the  simulated  ship  from  the 
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initial  position  to  the  final  position  while  avoiding  each  simulated  contact 
(other  ship)  encountered.  The  distance  between  the  initial  and  final 
positions  is  approximately  30  miles  and  the  time  allotted  for  the  transit 
is  90  minutes. 

The  subject  sits  in  front  of  a simulator  console  which  has 
three  computer  driven  CRT  displays  and  a number  of  control  switches. 
The  CRT  display  directly  in  front  of  the  subject  is  the  main  display 
and  provides  visual  information  about  the  location  of  own  ship  and 
contacts.  The  configuration  of  that  display  varies  with  the  particular 
display  function  simulated,  and  thus,  is  an  experimental  variable.  The 
CRT  display  on  the  subject's  left  provides  a list  of  contact  information 
such  as  location,  speed,  heading,  and  closest  point  of  approach  (CPA). 
This  display  is  termed  "LIST".  The  display  on  the  subject's  right 
provides  data  on  ship  course  and  speed.  It  can  also  provide  data  on 
"trial"  course  and  speed  which  the  subject  may  input  for  evaluating 
decision  command  changes.  There  are  a number  of  control  switches 
which  allow  the  subject  to  select  display  functions,  to  input  trial  course 
and  speed,  and  to  direct  course  and  speed  change  commands  to  an 
individual  representing  the  Helmsman. 

The  task  of  the  subject  OOD  is  to  visualize  or  calculate 
the  possible  courses  from  present  ship  position  to  the  objective  point. 
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Next,  by  taking  into  account  the  contact  positions  and  velocities  and  by 
utilizing  the  various  display  functions  available,  he  is  to  direct  the  ship 
to  the  objective  location.  The  instructions  read  to  the  subject  prior 
to  each  trial  are:  "the  objectives  during  the  run  are  to  be  at  the 

rendezvous  at  the  end  of  the  90  minute  period,  to  pass  clear  of  all 
contacts  (if  possible  by  more  than  4,000  yards),  to  obey  the  Rules  of 
the  Road,  and  to  observe  economy  of  operation." 

Performance  with  three  types  of  displays  is  of  interest. 

The  three  types  are  termed: 

1 . "OLD  system" 

2.  "PACS  system" 

3.  "RVV  system" 

A more  detailed  description  of  the  displays  is  presented  in  the  Appendix. 
The  OLD  system  is  essentially  a conventional  radar  display  where  the 
center  of  the  screen  is  the  position  of  own  ship  and  the  relative  positions 
of  contacts  are  shown  as  blips. 

PACS  (Probable  Area  of  Collision)  is  a new  display  providing 
a number  of  different  display  functions  which  are  selectable  by  the  subject. 
PACS  is  the  name  of  the  function  and  the  name  of  the  display  system 
providing  that  function.  In  PACS,  LIST  is  available  on  the  left-hand 
CRT.  Also,  the  subject  can  select  the  new  display  function  PACS,  or 
True  Velocity  Vector  modes  on  the  center  CRT.  With  the  PACS  function. 
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the  display  provides  the  locus  of  collision  points  for  each  contact  for 
all  possible  own  ship  courses.  Since  this  requires  projection  of  future 
own  ship  and  contact  positions,  the  display  aids  the  subject  in  selecting 
a new  course.  With  the  true  velocity  vector  mode,  velocity  vectors 
are  superimposed  both  on  the  own  ship  and  contact  blips  to  show  the 
predicted  position  of  all  the  ships  at  a future  time  selected  by  the  operator. 

Finally,  another  new  display  termed  the  "RVV  system" 
includes  LIST  plus  relative  velocity  vectors.  With  this  display  type, 
relative  velocity  vectors  are  superimposed  on  contact  blips  indicating 
the  relative  position  of  each  contact  with  respect  to  own  ship  at  a future 
time  selected  by  the  operator. 

The  experiment  design  required  1 6 subjects  to  perform  with 
each  of  the  three  display  designs.  This  design  is  a two  way  experiment 
with  repeated  measures  on  one  factor.  Two  display  orientations.  Head-up 
and  North-up,  were  controlled  in  the  experiment  by  randomly  selecting 
subjects  for  each  orientation.  In  addition,  the  following  factors  were 
selected  at  random: 

• Problem  orientation  (the  path  to  objective  location 
is  at  0°  or  120°  from  the  North) 

• Problem  difficulty  (three  variations  of  contact  speeds) 

• Sequence  of  display  designs  presented  to  the  subject 
A chart  summarizing  the  design  of  the  experiment  is  given  in  Table  1 . 
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HEAD-UP  NORTH-UP 


)A,B,C  refers  to  the  sequence  with 

which  displays  are  presented  to  subject. 

I 

TABLE  1 CONTROL  TASK  CONDITIONS  FOR  EACH 

SSBCS  EXPERIMENT  TRIAL 


2-6 


MPMaw 


2.2  Concepts  and  Analytical  Tools 

2.2.1  Method  of  Identifying  an  OMAC  to  Represent 

Subject  Controlled  Ship  Responses 

As  described  in  the  introduction,  the  analysis  of  the  effect 
of  display  design  on  ship  responses  (i.e.,  ship  motion  under  subject 
control  from  the  starting  location  to  the  objective  location)  employs  a 
constraint  which  uses  optimal  control  theory.  With  control  theory 
the  OMAC  which  models  ship  responses  is  determined. 

The  basic  problem  in  identifying  an  OMAC  is  that  optimal 


I 

1 


control  theory  is  a synthesis  process  which  starts  with  criteria  selection 
and  results  in  identification  of  optimal  responses,  i.e.,  first  the  criteria 
are  selected  and  constraints  are  identified,  then  each  possible  response 
that  satisfies  the  constraints  is  evaluated  and  the  one  judged  best  according 
to  the  criteria  is  identified.  The  theory  provides  an  efficient 
computational  algorithm  for  identifying  the  best  response.  However,  the 
proceduee  desired  here  is  the  inverse  of  that  synthesis  process.  The 

\ 

desired  process  starts  with  the  ship  responses  generated  in  experiments 
using  OQD's  to  control  the  ship,  and  is  to  result  in  the  identification  of 
the  criteria  optimized.  Thus  control  theory  does  not  provide  the  exact 
process  required.  But  an  equivalent  to  the  inverse  process  is  obtained 
if  a set  of  candidate  criteria  are  selected  and  the  associated  optimal  ship 
response  for  each  criteria  is  determined.  (Note  that  the  phrase  "optimal 
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ship  response"  does  not  mean  best  ship  response  in  an  absolute  sense, 
but  only  best  response  with  respect  to  associated  candidate  criteria.) 

Each  of  the  optimal  ship  responses  can  be  compared  with  one  of  the 
ship  responses  obtained  from  the  experiment  subject  trials.  Where 
close  agreement  is  found  between  the  optimal  ship  response  and  the 
subject  trial  response,  the  associated  candidate  OMAC  is  identified 
as  being  optimized  by  the  subject  trial  response.  This  process  is  an 
approximation  to  the  desired  inverse  process.  It  is  illustrated  in  Figure  1. 

The  comparison  of  optimal  and  subject  trial  responses  is 
a test  of  the  similarity  of  their  contact  avoidance  performance.  The 
transition  performance  measure  validated  in  Analysis  I is  used  to 
measure  that  contact  avoidance  performance.  A description  of  the 
measure  is  given  in  Section  2.2.2.  Section  2.2.3  provides  a 
description  of  the  method  of  comparison  using  the  measure. 

As  outlined  above,  a set  of  candidate  OMAC's  is  formed 
as  part  of  the  process  to  identify  the  OMAC  optimized  by  a ship  response 
obtained  in  the  experiment.  These  candidate  OMAC's  are  formed  using 
a sum  of  weighted  variables  identified  in  the  instructions  given  to  the 
subjects.  Specifically,  the  variables  used  are  transit  time  and  minimum 
passing  distance  from  each  contact.  A set  of  weighting  values  is 
selected  to  form  a set  of  candidate  OMAC's  as  described  in  Section  2.2.4. 
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Candidate  OMAC's 


The  comparison  of  performance  of  subjects  using  different 
displays  with  the  performance  of  OMAC's  is  illustrated  in  Figure  2. 
In  this  case,  ship  response  data  from  the  subjects  using  a display 
design  are  combined  for  the  performance  analysis  (Section  2.2.2) 
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Contact  Avoidance  Performance  Measurement 


In  a previous  report  (Connelly  1976),  the  statistical 
significance  of  performance  differences  with  different  display  types 
was  demonstrated.  Specifically,  it  was  shown  that  contact  avoidance 
performance  with  the  PACS  type  of  display  provided  a statistically 
significant  increase  in  the  safety  margin  compared  to  performance 
with  the  RVV  and  OLD  displays.  The  performance  measure  used 
to  obtain  this  result  employed  a count  of  the  number  of  occurrences, 
in  each  experimental  trial,  that  critical  CPA1  values  were  projected 
to  occur  within  the  next  15  minutes.  This  measure,  although  crude, 
revealed  the  performance  differences  with  different  displays  mentioned 
above.  However,  this  measure  only  reveals  the  number  of  occurrences 
of  critical  CPA  values.  It  does  not  reveal  the  patterns  of  CPA  values 
occuring  in  time  with  each  display  type.  In  order  to  be  able  to  identify 
such  patterns,  a dynamic  contact  avoidance  measure,  termed  "transition 
measure,"  was  developed  and  is  described  in  the  following  paragraphs. 


*CPA  is  closest  point  of  approach  and  TCPA  is  time  to  CPA. 
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Candidate  OMAC's 


FIGURE  2 CO/ PAR  IS  ON  OF  PERFORMANCE  WITH  OMAC'S  TO  PERFORMANCE 

OF  SUBJECTS  WITH  EACH  DISPLAY  DESIGN 


Contact  Avoidance  Maneuvering 

CPA/TCPA  Matrix 

According  to  the  instructions  provided  to  the  operator, 
he  is  to  direct  the  ship  to  the  objective  point  within  the  prescribed 
time  while  attempting  to  avoid  contacts  by  at  least  4,000  yards. 
Analysis  of  how  he  accomplishes  the  task  requires  a method  of 
dividing  the  problem  situation  into  categories  so  that  performance 
in  each  category  can  be  examined.  The  selected  method  uses  a 
CPA/TCPA  matrix  as  shown  in  Figure  3.  Cells  in  the  matrix 
correspond  to  intervals  of  CPA  and  TCPA.  At  each  time  interval, 
the  state  of  the  ship  (position,  speed,  course)  and  state  of  each 
contact  specify  a cell  condition.  For  N contacts  there  are  N cell 
conditions.  For  example.  Cell  1 corresponds  to  a projected  CPA 
of  from  0 to  1.5  miles  which  is  projected  to  occur  within  0 to  5 
minutes.  Cell  2 corresponds  to  the  same  CPA  but  is  projected  to 
occur  within  5 to  10  minutes.  Cell  21  represents  a different  type 
of  situation  where  the  CPA  has  already  occurred,  i.e.,  range  to  the 
contact  is  increasing.  Projected  CPA  violations  (passing  within  4,000 
yards  (2  miles)  ) are  identified  by  entries  in  Cells  1 through  12  and 
Cell  19. 
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Time-to-CPA  (TCPA)  (Mi 


Closest  Point  of  Approach  (CPA) 
(Miles) 


CPA  Violation  ► 

1.5  2.0  2.5 


*Cells  1,  2,  3 and  7,  8,  9 are 
designated  critical  cells  where 
a CPA  violation  is  projected  to 
occur  within  15  minutes. 


FIGURE  3 DEFINITIONS  OF  CPA/TCPA  CELLS 


As  an  aid  to  understanding  how  the  entries  in  the  CPA/ 

TCPA  matrix  change  as  a function  of  ship  and  contact  position  changes, 
consider  the  following  typical  situations. 

If  the  ship  speed  and  course  remain  constant  over  several 
time  intervals,  cell  conditions  change  to  decrease  TCPA.  For  example, 
an  entry  in  Cell  6 would  "transfer"  to  Cell  5 because  of  the  elapsed  time. 

If  ship  course  or  speed  is  changed  by  the  subject  OOD  as  a 
contact  avoidance  or  transiting  maneuver,  other  cell  transitions  can 
occur.  As  an  example,  an  entry  in  Cell  6 identifies  a condition  where, 
unless  own  ship  (or  contact)  course  or  speed  is  modified,  the  two  ships 
will  pass  within  4,000  yards  in  30  minutes.  If  no  action  is  taken,  the 
entry  in  Cell  6 will  transfer  to  Cell  5 because  of  the  elapsed  time.  In 
order  to  avoid  a contact  violation  (passing  within  4,000  yards),  it  is 
necessary  that  the  subject  OOD  command  a change  of  ship  speed  or 
course  such  that  the  entries  are  transferred  to  a cell  without  a projected 
violation. 

Transition  Matrix 

Cell  transition  patterns  characterize  the  subject  OOD's 
contact  avoidance  and  transit  maneuvering.  Similarly,  transition 
patterns  representing  a group  of  OOD's  (such  as  those  using  a particular 
display  design)  can  be  used  to  characterize  the  group  maneuver  patterns. 
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Cell  transitions  are  represented  by  transition  probabilities 
since  cell  sequences  obtained  from  experimental  data  are  not  always 
the  same.  This  is  due  to  performance  variations  in  individuals  and 
among  individuals.  The  transition  pattern  from  a given  celt,  for 
example  Cell  i,  is  recorded  in  the  associated  row  of  a 21  by  21 
transition  matrix  representing  all  possible  transitions.  Probability  (p. .} 
is  the  probability  of  transferring  to  Cell  j from  Cell  i.  Probabilities 
along  the  matrix  diagonal  (where  i = j)  do  not  describe  cell  sequences 
and  have  no  meaning  in  this  analysis.  As  a result,  the  diagonal  matrix 
elements  are  zero. 

Although  the  above  description  of  the  CPA/TCPA  matrix, 
the  transition  matrix,  and  the  method  of  calculating  transition  probabilities 
is  presented  as  though  cell  entry  changes  occur  for  only  one  contact  at 
a time,  there  are  in  fact  multiple  simultaneous  changes.  Each  course 
or  speed  change  commanded  by  the  OOD  can  cause  multiple  simultaneous 
changes  in  the  cell  entries  because  of  the  existence  of  multiple  contacts. 
Transition  probabilities  are  developed  from  the  transitions  that  occur 
both  simultaneously  and  sequentially  throughout  an  experimental  trial. 

When  a transition  matrix  is  developed  to  represent  performance  of  a 
group  of  subjects,  data  used  are  obtained  from  all  subject  runs.  In 
spite  of  the  fact  that  the  transition  matrix  probabilities  represent  both 
simultaneous  and  sequential  occurrences  and  multiple  subject  runs,  it 
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is  often  convenient  to  visualize  the  transition  matrix  as  describing 
the  relationship  between  own  ship  and  one  contact  - in  the  presence 
of  many  other  contacts.  With  this  visualization,  it  is  possible  to 
trace  the  cell  sequences  from  the  point  of  contact  activation  (contact 
appearance  on  the  display)  until  the  ship  reaches  the  objective  point. 
Different  contact  initial  conditions  (conditions  at  time  of  contact 
activation)  may  lead  to  different  cell  sequences. 

As  a means  for  illustrating  both  cell  transition  matrices 
and  the  different  cell  transition  patterns  obtained  with  different  types 
of  display,  three  transition  diagrams  have  been  prepared.  These 
diagrams,  shown  in  Figures  4,  5,  and  6,  are  developed  from  transition 
matrices  shown  in  Figures  7,  8,  and  9 for  the  PACS,  OLD  and 
RVV  displays  respectively.  Only  transitions  with  a probability  greater 
than  0.10  and  transitions  from  Cells  1 through  5 are  illustrated. 

From  the  transition  diagrams  the  superior  contact 
avoidance  performance  obtained  with  the  PACS  display  is  easily 
seen.  For  example,  with  the  PACS  display  the  probability  of 
transition  from  Cell  5 to  cells  where  no  CPA  violation  is  projected 
is  0.35  (.22  + .12).  This  represents  superior  contact  avoidance 

maneuvering  as  compared  to  the  corresponding  probabilities  of  .17 
and  .16  for  the  OLD  and  RVV  displays  respectively.  Superior 
transition  patterns  for  PACS  also  exist  for  Cells  4 through  1 as  illustrated 
in  the  Figures. 


■ 
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FIGURE  6 MAJOR  TRANS 

WITH 
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FIGURE  9 CELL  TRANSITION  MATRIX  FOR  RVV  DISPLAY 


These  diagrams  provide  convincing  graphical  evidence  of 
the  performance  differences  obtained  with  different  displays.  Quantitative 
measurement  of  those  differences,  as  a step  in  identifying  the  reason 
for  those  differences,  is  described  in  the  next  section. 

2.2.3  Comparison  of  Ship  Response  Patterns 

Considerable  analysis  of  ship  maneuvering  patterns  can 
be  accomplished  by  inspection  of  the  transition  matrix  probabilities. 

For  example,  an  entry  in  Cell  6 corresponds  to  a projected  CPA  violation 
in  25-30  minutes.  Transitions  to  Cells  5,  4,  3,  2,  1 will  occur  if  no 
OOD  control  actions  are  taken  to  cause  a transfer  to  cells  where  no 
CPA  violation  is  projected.  Also,  as  time  to  CPA  decreases  in  cells 
where  a CPA  violation  is  projected,  the  situation  becomes  less  desirable. 
Thus,  Cell  1 is  less  desirable  than  Cell  2 which  is  less  desirable  than 
Cell  3 and  so  forth.  Therefore,  examination  of  transition  probabilities 
provides  a way  of  evaluating  contact  avoidance  performance  and  also 
comparing  performance  with  different  experimental  treatments. 

One  difficulty  in  the  quantitative  evaluation  of  performance 
by  comparing  transition  probaoilities  directly  is  that  transitions  from  a 
given  cell  are  not  weighted  by  the  probability  that  the  cell  is  "used." 

For  example,  with  one  treatment  (one  display  type),  transitions  from 
Cell  3 may  be  better  (a  higher  probability  of  transfer  to  cells  that  do 
not  project  a CPA  violation)  than  that  of  another  treatment. 


But  the  probability  that  conditions  corresponding  to  Cell  3 exist  during 
the  experiment  may  be  so  small  that  transitions  from  Cell  3 may  be  of 
little  consequence.  A method  for  solving  this  problem  is  described  in 
the  next  paragraph  following  introduction  of  a required  concept. 

Once  a contact  is  passed,  where  the  distance  to  the  contact 
is  increasing,  there  is  little  likelihood  that  own  ship  will  maneuver  so 
as  to  approach  that  contact  again.  Assuming  that  restriction  is  true. 

Cell  21  can  be  treated  as  an  absorbing  cell,  i.e.,  the  probability  of 
leaving  Cell  21  after  it  is  entered  is  zero.  With  that  assumption,  the 
transition  matrix  can  be  analyzed  as  an  absorbing  Markov  Process 
(Kemeny  & Snell,  1960).  This  means  that  the  process  is  considered  as 
starting  in  any  cell  (but  especially  Cells  19  and  20)  and  transferring  to 
a series  of  other  cells  ending  in  Cell  21  . The  process  may  return  to 
a given  cell  in  that  process  but  once  it  enters  Cell  21,  it  remains  there. 
The  analysis  of  interest  is  the  calculation  of  the  mean  number  of  times 
each  cell  will  be  entered  before  entering  the  absorbing  state,  given  the 
starting  cell.  This  provides  the  desired  weighting  of  cell  usage.  The 
contact  avoidance  measure  of  interest  is  the  mean  number  of  times 
critical  Cells  1,  2,  3,  and  7,  8,  9 are  entered  given  the  process  starts 
in  Cells  5,  11,  and  17. 
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Calculation  of  the  measure  requires  formulation  of  a matrix 
(Q)  which  is  identical  to  the  transition  matrix  P except  that  Row  21  and 
Column  21  (corresponding  to  the  absorbing  cell)  have  been  removed. 

The  desired  measure  is  given  by  matrix  R (a  20  x 20  matrix): 

R = (1  - Q)-1 

where  I is  the  identity  matrix,  "-1"  represents  the 
matrix  inverse  operation,  and  elements  r of  R are 
the  mean  number  of  times  the  process  enters  Cell  j 
given  the  process  starts  in  Cell  i. 

Since  the  measure  of  interest  is  the  mean  number  of  times  that  the 
process  enters  the  critical  cells,  given  the  process  starts  in  Cell  5, 
plus  those  means  given  the  process  starts  in  Cell  11  , and  again  for 
Cell  17,  the  specific  performance  measure  of  interest  is 

p = 2 

m i=5,  11,  17 

j=1 , 2,  3,  7,  8,  9 

Thus,  the  measure  includes  six  values  for  each  of  three  initial  conditions 
for  a total  of  18  terms. 

When  a comparison  of  the  contact  avoidance  performance 
is  made  (i.e.,  comparing  performance  of  a subject  using  two  different 
displays  or  comparing  performance  of  a subject  with  the  performance  of 
an  OMAC)  the  differences  in  the  mean  number  of  times  in  the  critical 
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r = mean  number  of  times  in  Cell  i 

lij 

given  that  the  initial  cell  is  Cell  j for  Case  1 
r ..  = mean  number  of  times  in  Cell  i 

2ij 

given  that  the  initial  cell  is  cell  j for  Case  2 
2.2.4  Candidate  QMAC's 

OMAC's  are  defined  by  the  parameters  of  the  criteria  and 
constraint  functions.  The  following  criterion  function  was  selected  to 
include  tne  factors  identified  as  important  in  the  task  instructions. 

C = AxT  + Bxf(2-  CPA)/TCPA 

where 

T is  total  transiting  time, 
f is  a function  defined  by  Figure  10, 

CPA  is  closest  point  of  approach  to  a contact,  and 
TCPA  is  time  to  CPA. 
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FIGURE  10  CRITERION  FUNCTION 
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Weighting  coefficients  A and  B are  limited  as  follows: 

A,  B > 0.0 
A + B = 1 . 

In  addition  to  the  criterion  function  given  above,  a purview  constraint 
is  employed.  A purview  constraint  requires  that  only  contacts  within 
the  purview  range,  which  is  specified  in  mites,  can  be  used  to  develop 
ship  controls.  This  consideration  of  only  those  contacts  within  purview 
range  is  maintained  eventhough  the  process  of  developing  the  ship  control 
using  OMAC  requires  planning  a ship  response  beyond  the  purview  range 
to  the  objective  point. 

Table  2 provides  a list  of  the  criterion  and  purview 
parameters  values  employed.  These  parameter  values  define  the  set 
of  candidate  OMAC's  tested. 

Figure  11  is  a chart  indicating  the  performance  obtained 
as  a function  of  OMAC  parameters.  Performance  is  measured  with  the 
transition  measure  described  in  Section  2.2.2.  As  shown  in  the  figure, 
performance  improves  with  increasing  purview  up  to  12.5  miles.  At 
the  greater  purview  tested  (16\67  miles),  performance  decreased. 
Performance  with  a purview  of  16.67  rmles  was  evaluated  with  two  time 
weighting  values  (-,01  and  .25).  As  shown  in  Figure  11,  a time  weighting 
of  .01  results  in  poor  overall  performance.  With  this  result  that  weighting 
was  not  evaluated  with  other  purview  values.  But,  the  16.67  mile  purview 
was  evaluated  with  a time  weighting  of  .25.  The  figure  shows  performance 


2-28 


..  


OMAC 

IDENTIFICATION 


PURVIEW 
RANGE  (MILES 


O/ViAC  1 


TABLE  2 CRITERION  AND  PURVIEW  PARAMETERS 


purview  required  contact  avoidance  maneuvering  far  in  advance  of  that 
required  by  the  performance  measure.  With  this  excessive  contact 
avoidance  maneuvering,  an  excessive  time  penalty  is  accumulated. 
Apparently  the  12.5  mile  purview  does  not  require  excessive  contact 
avoidance  maneuvering  and  therefore  provided  best  performance. 

2.3  Method  of  Analysis 

2.3.1  Analysis  1:  Test  of  the  Dynamic  Performance  Measure 

The  transition  performance  measure  is  described  in 
Section  2.2.2.  It  is  obviously  related  to  the  "static"  measure,  i.e., 
the  count  of  the  number  of  times  critical  cells  are  entered;  and,  prior 
to  its  use,  it  is  desirable  to  test  the  relationship  between  the  static 
and  transition  measures.  This  test,  termed  Analysis  1,  employs  a 
regression  analysis  of  subject  performances  with  the  static  measure  as 
the  independent  variable  and  the  transition  measure  as  the  dependent 
variable.  Two  analyses  are  used.  One  employs  data  from  subject 


performances  using  the  PACS  display  and  the  other  uses  data  obtained 
with  the  OLD  display.  The  hypothesis  is  that  the  two  measures  indicate 
the  same  quantity. 

2.3.2  Analysis  II:  OMAC  Representation  of  Subject  Performance 

Contact  avoidance  performance  of  each  OMAC  is  compared 
to  the  contact  avoidance  performance  of  each  subject  experimental  trial 
in  order  to  identify  a representative  OMAC  for  that  trial . The  OMAC 
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with  the  performance  closest  to  a subject  trial  (as  indicated  by  the 


transition  performance  measure)  is  designated  as  the  representative 
OMAC  for  that  subject. 

A test  of  that  representation  was  made  using  a regression 
analysis  where  the  performance  of  a representative  OMAC  is  used  to 
predict  the  performance  of  a subject  using  a particular  display  type. 

The  analysis  uses  OMAC  contact  avoidance  performance  as  the  independent 
variable.  The  hypothesis  is  that  OMAC  contact  avoidance  predicts  subject 
performance  with  a specified  display  type. 

2.3.3  Analysis  III:  OMAC  Parameters  Representing  Display  Type 

OMAC’s  representing  OOD  performance  can  be  grouped 
according  to  the  display  type  (design)  used.  Parameters  of  OMAC's 
identified  for  each  display  type  can  be  analyzed  to  explain  performance 
differences  with  different  display  types.  OMAC  parameters  for  two 
display  types  (PACS  and  OLD)  are  analyzed. 

In  addition  to  display  type,  OMAC  parameters  are  affected 
by  the  subject  performance  level.  In  order  to  study  this  effect  in 
combination  with  the  effect  of  display  type,  a two  variable  analysis 
is  used  with  two  levels  per  variable.  A possible  third  factor,  accuracy 
of  OMAC  models  in  representing  subject  performance,  is  not  used  in  the 
analysis  because  the  accuracy  of  the  OMAC  representation  (i.e.,  the 
difference  between  OOD  performance  and  the  performance  of  the  OMAC 
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selected  as  representative)  is  correlated  with  OOD  performance. 

This  relationship  is  illustrated  in  Table  3. 

In  order  to  establish  two  performance  categories  for 
subjects,  the  superior  half  of  the  subject  trials  with  the  OLD  display  are 
classified  as  superior  and  the  other  half  as  less  than  superior.  With  this 
classification,  the  performance  score  for  the  superior  subject  trials 
was  .526  or  less.  This  score  level  was  then  taken  as  the  superior 
performance  criterion  level  and  applied  to  identify  superior  performances 
with  the  PACS  display.  With  this  selection  criteria,  nine  of  the  total 
11  subject  trials  with  the  PACS  display  were  rated  superior. 

From  this  two  factor  analysis,  estimates  are  developed 
of  OMAC  parameters  for  each  display  type  and  performance  category. 

Finally,  the  OMAC  parameters  are  analyzed  using  a t_  test  for 
significant  differences  with  display  type. 

2.3.4  Analysis  IV:  Display  Information  and  Ship  Control  Performance 

Each  type  of  display  presents  information  to  the  subject  in  a 
different  form.  The  information  and  the  form  of  the  information  in 
advanced  displays  such  as  the  PACS  display  can  eliminate  the  need  for 
the  OOD  to  perform  certain  information  processing  tasks.  The  OOD  is 
then  presumably  free  to  perform  other  processing  tasks.  Relating  this 
change  of  the  information  processing  requirement  to  a change  in  contact 
avoidance  performance  is  the  purpose  of  this  analysis. 
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Difference  in 
Scores  Between 
Subject  Perfor- 
mance and  OMAC 
Representing 
Subject  Performance 


Subject 

Performance 

Score 


1 .389 


1 .222 


1 .966 


2.900 


5.061 


TABLE  3 ORDERING  OF  PERFORMERS 
WITH  THE  OLD  DISPLAY 
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The  portion  of  the  OOD's  ship  control  task  that  is  expected 
to  be  affected  by  a display  design  is  the  information  processing  component. 
This  component  is  composed  of  a set  of  subtasks  which  are  taken  as: 
locate  contact  on  display,  determine  contact  course  and  speed,  project 
contact  position  as  a function  of  future  time,  identify  possible  CPA 
violations,  determine  time  to  CPA  for  CPA  violations,  evaluate  alternative 
speeds  to  avoid  CPA  violations,  and  select  course  and  speed.  Each 
display  design  can  provide  information  that  can  simplify  or  even 

supplement  one  or  more  of  these  information  processing  subtasks.  For 

1 

analysis  in  this  study,  specific  subtasks  are  categorized  as  follows: 

1 . Determine  course  and  speed  required  to  reach 
objective  position  independent  of  contact 
positions,  courses,  and  speeds. 

2.  Detect  CPA  violations  and  estimate  time  to  CPA 

3.  Identify  alternative  courses  and/or  speeds  that 
will  avoid  CPA  violations. 

4.  Select  alternative  course  and/or  speed  to  avoid 
CPA  violation  based  on  transiting  objectives,  to 
obey  rules  of  the  road,  and  to  conserve  fuel. 

5.  Perform  other  information  processing  tasks  not 
related  to  contact  avoidance. 
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The  analysts  is  to  identify » for  each  display  type,  the 


processing  provided  by  the  display.  In  sun  additional  analysis,  the 
average  number  of  contacts  within  purview  is  determined  in  order 
to  establish  the  average  number  of  contacts  processed  as  a function 
of  purview. 
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3.0  RESULTS 

3.1  Analysis  1 Results:  Test  of  the  Dynamic  Performance  Measures 
Results  of  the  regression  analysis  using  the  transition  and 

static  performance  measures  are  given  in  Tables  4 and  5.  The  results 
show  that  the  static  measure  predicts  the  transition  measure  and  that  the 
prediction  is  significant  at  the  a < .001  and  .05  levels  for  the  PACS 

and  OLD  displays  respectively. 

3.2  Analysis  II  Results:  OMAC  Representation  of  Subject  Performance 
Tables  6a  and  6b  give  the  results  of  the  comparison  of  each 

subject  trial  performance  with  each  OMAC . Each  subject  provided 
two  performances  - one  using  the  PACS  display  and  the  other  using 
the  OLD  display.  Entries  indicated  by  a identify  the  representative 
OMAC  selected . 

Results  of  the  comparison  between  the  representative  OMAC 
and  the  represented  subject  trial  according  to  contact  avoidance  performance 
are  shown  in  Tables  7 and  8.  The  regression  is  significant  at  the 
a < 0.001  level  for  both  PACS  and  OLD  displays.  A scatter  diagram 

for  these  regressions  is  given  in  Figure  12  . 

3.3  Analysis  III  Results:  OMAC  Parameters  Representing  Display  Type 
Table  9 presents  the  results  of  the  analysis  of  OMAC  parameters 

as  a function  of  all  performances  (Case  I)  and  superior  performances 
(Case  II),  and  two  display  types  (OLD  and  PACS). 
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SOURCE  OF 
VARIATION 

SS 

df 

MS 

F ratio 

Explained  - due  to 

linear  regression 

/N  — 

Y - Y 

7.92 

1 

7.92 

12.68* 

Unexplained  - error 
around  regression 
line 

5.61 

9 

0.62 

TOTAL 

13.53 

10 

F(.001,  1,9)  = 9.5* 

Significance  of  Regression  Coefficient 

t = 45.2,  t(.001,  9)  = 4.7 

Regression  Coefficient  ^ = 0.280 

V intercept  a - 0.092 


TABLE  4 REGRESSION  ANALYSIS 
PACS  DISPLAY 


TABLE  6a  COMPARISON  OF  SUBJECT  AND  OMAC  PERFORMANCE 


PACS  DISPLAY 


SOURCE  OF 
VARIATION 


Explained  - due  to 
linear  regression 


Unexplained  - error 

l 

around  regression 

i 

I 

line  ! 


3.16 


df 

MS 

1 

10.37 

9 

.35 

F ratio 


29.50 


TOTAL 


13.53 


Significance  of  Regression  Coefficient 


t = 5.4  , t(.001,  9)  = 4.7 

s 


Regression  Coefficient  b =3.0 

y • x 

Y intercept  a = -0.1 


TABLE  7 REGRESSION  ANALYSIS  OMAC/SUBJECT 
CONTACT  AVOIDANCE  PERFORMANCE 
PACS  DISPLAY 


SOURCE  OF 
VARIATION 


SS 


df 


MS 


F ratio 


Explained  - due  to 


linear  regression 

l/N  — 

V - V 


20.83 


20.83 


50. 1 4* 


Unexplained  - error 
around  regression  I 3.73 

line 

TOTAL  - 24.56 


10 


.41 


F(.001,  1,  9)  = 22.9* 

Significance  of  Regression  Coefficient 

t = 7.08,  t(.001,  9)  = 4.78 

Regression  Coefficient  . x = 1 *41 

Y intercept  a = 1.02 


TABLE  8 REGRESSION  ANALYSIS  OMAC/SUBJECT 
CONTACT  AVOIDANCE  PERFORMANCE 
OLD  DISPLAY 


ij 
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PURVIEW 

WEIGHTING  ON 
TRANSIT  TIME 

OLD 

PACS 

Significance 
test  ts 
Ratio 

OLD 

PACS 

10.20 

11  .33 

0.84  (NS) 

0.211 

0.15 

9.56 

9.49 

0.18 

0.006 

9.60 

12.00 

2.51* 

0.15 

0.15 

4.05 

0.85 

significant 
at  a < . 1 

0.008 

0.007 

CASE  I 


Mean 

Variance 

CASE  II 


Mean 

Variance 


V (o  = .1)  = 2.096 

t'  (a  = .05)  = 2.717 

Case  I - All  subjects  using  both  OLD  and  PACS  display 

Case  II  - Only  subjects  providing  superior  performances 


TABLE  9 OMAC  PARAMETERS  vs.  DISPLAY  TYPE 

AND  PERFORMANCE 
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Results  show  that  the  time  weighting  parameter  (A)  (and 


therefore  the  weightings  on  contact  avoidance  (B)  since  B = 1 - A) 

are  invariant  with  display  type  for  superior  performers.  Both  mean 
(.15)  and  variance  (»  .008)  are  the  same  for  OLD  and  PACS  displays. 

When  less  than  superior  performers  are  considered  (which 
occurs  more  frequently  with  the  OLD  display)  the  time  weighting  increases 
with  the  OLD  display  but  remains  stationary  with  the  PACS  display. 

Considering  the  purview  parameter  and  superior  performers, 
the  OLD  display  has  a rating  of  9.6  miles  while  the  PACS  display  has 
a rating  of  12.0  miles.  This  purview  difference  is  significant  at  the 
a < 0.1  level.  The  associated  variance  is  larger  for  OLD  (4.05)  than 
for  PACS  (0.85). 

Analysis  of  purview  with  all  performances  shows  an 
increase  in  variance  (»9.5)  for  both  display  types.  Also,  the  mean 
purview  tends  to  decrease  with  the  PACS  display  and  increase  with  the 
OLD  display. 

3,4  Analysis  IV  Results:  Display  Information  and  Ship  Control  Performance 

Results  of  the  task  allocation  analysis  are  shown  in  Table  10. 
According  to  the  analysis,  the  OOD  must  provide  all  of  the  information 
processing  tasks  identified  when  using  the  OLD  display.  In  contrast, 
the  PACS  display  provides  virtually  all  processing  required  to  perform 
Tasks  2 and  3,  detection  of  potential  CPA  violations,  and  identifying 
candidate  alternative  causes  and  speeds  to  avoid  the  CPA  violation. 


TASK 

ALLOCATION 

OLD  DISPLAY  PACS  DISPLAY 

TYPE 

TASK 

ID 

1 . 

Determine  course 

OOD** 

OOD 

T1 

and  speed  to 

objective 

2. 

Detect  CPA  violations 

OOD 

D* 

t 

1 

and  determine  time 

to  violation 

3. 

Identify  alternative 

OOD 

D* 

course  and  speeds  to 

avoid  CPA  violations 

4. 

Select  alternative 

OOD 

OOD 

*3 

5. 

Other  information 

OOD 

OOD 

T2 

processing 

•Display  provides  substantially  the  total  effort  required  by  these  tasks. 

"""Tasks  requiring  information  processing  by  the  Officer  of  the  Deck  (OOD). 

Lower  case  t represents  tasks  where  the  information  processing  required 
is  a function  of  the  number  of  contacts. 

Upper  case  T represents  tasks  where  the  information  processing  required 
is  not  a function  of  the  number  of  contacts. 

TABLE  10  INFORMATION  PROCESSING  TASK  ALLOCATION 
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Analysis  of  the  average  number  of  contacts  in  purview  as 
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a function  of  purview  reveals  that  the  average  number  of  contacts  is 
proportional  to  area  within  purview.  This  implies  that  a constant 
contact  density  can  be  used  to  predict  number  of  contacts  in  a given 
purview.  This  result  is  shown  in  Figure  13.  The  contact  density 
for  the  ship  control  problem  is  approximately  0.012  contacts  per 
square  mile. 
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4.0 


DISCUSSION  AND  CONCLUSIONS 


Products  of  the  Research 


Performance  Measurement 


Results  of  the  comparison  of  the  transition  and  static  measures 
(Analysis  I)  provide  convincing  evidence  of  the  validity  of  both  the  transition 
measure  and  the  transition  matrices  used  to  compute  the  transition  measure. 
In  this  study,  the  transition  measure  was  used  extensively  to  compare  OMAC 
performance  to  subject  performance.  This  validation  of  the  measure  is 
fundamental  to  the  other  results  obtained  in  the  study.  The  transition 
matrix  probabilities  reveal  the  nature  of  the  performance  differences  obtained 
with  different  display  types.  Analysis  of  the  transition  probabilities  was 
used  in  this  study  to  illustrate  the  nature  of  the  performance  differences. 

A more  extensive  analysis  could  reveal  control  pattern  tendencies  for  each 


display  type. 

The  transition  performance  measure  is  a sensitive  indication 
of  contact  avoidance  performance.  Its  sensitivity  comes  from  detection 
of  the  relative  ship  states  that  lead  to  collisions  rather  than  detection  of 
just  collision  or  near  collisions. 

OMAC  Representation  of  Subject  Performances 

Analysis  II  results  identify  an  OMAC  for  each  subject  per- 
formance. Two  factors  are  of  immediate  interest  with  this  identification. 
First,  the  ability  of  the  OMAC's  to  represent  subject  performances  is  a 
function  of  the  level  of  subject  performance,  i.e.,  superior  performers 


ail  n -•  ■■  ■ - 


were  represented  more  accurately  than  were  lesser  performers.  This 
suggests  the  need  to  introduce  additional  factors  in  candidate  OMAC's 
in  c^der  to  more  accurately  represent  the  lesser  performers. 

A second  factor  of  interest  is  that,  in  spite  of  difference 
in  accuracy  of  representation,  the  representation  is  sufficiently  accurate 
to  permit  prediction  of  subject  performance,  i.e.,  prediction  of  subject 
performance  using  OMAC's  in  a statistically  significant  manner. 

GMAC  Representation  of  Performance  with  OLD  and 

PACS  Displays 

Analysis  III  is  an  analysis  of  OMAC's  representing  two 
different  display  types  (OLD  and  PACS)  for  two  subject  population 
groups  (all  subjects  and  superior  subjects).  From  the  results,  several 
conclusions  are  possible: 

1 . A greater  proportion  of  subject  performers  are  rated 
superior  with  the  PACS  display  than  with  the  OLD  display.  This  suggests 
that  with  the  PACS  display  there  would  be  more  superior  performers  in 
the  general  OOD  population. 

2.  Superior  performers  with  PACS  and  OLD  displays  are 
represented  by  the  same  OMAC  criteria  (the  relative  weighting  between 
contact  avoidance  and  transit  time).  The  average  performance  level  with 
the  two  displays  is  different  but  the  criteria  optimized  are  the  same.  This 
suggests  that  consistent  "target"  criteria  (the  criteria  sought  by  the  subject) 
was  the  goal  of  superior  performers  with  both  displays;  but,  the  subjects 


were  better  able  to  reach  that  performance  goal  with  the  PACS  display. 


1 

This  result  suggests  that  the  "target"  criteria  might  be  used  in  training 
where  subjects  could  be  rated  not  only  on  performance  but  also  on  the 
criteria  they  apparently  optimize. 

3.  Performance  differences  with  different  display  types 

obtained  from  the  superior  performances  are  explained  by  a difference 
in  purview  (the  range  from  own  ship  within  which  the  subject  processes 
contacts).  The  logic  is:  OMAC  predicts  subject  performance ; OMAC 

has  two  parts:  criteria  and  purview;  but,  the  criteria  is  constant  while 

purview  changes  with  changes  in  display.  Indeed,  the  OMAC  performance 
data  show  clearly  the  effect  of  purview  on  performance. 

4.  Since  the  average  number  of  contacts  within  purview 

in  the  experiment  problem  is  a linear  function  of  purview  area  (a  result 
of  Analysis  IV),  OOD  ship  handling  performance  may  be  a function  of 
the  number  of  contacts  that  the  OOD  can  process  with  a given  display 
design.  If  a display  feature  automates  one  or  more  information  pro- 

4 

cessing  tasks,  the  OOD  may  be  able  to  process  more  contacts,  thus 
expanding  h;s  purview.  According  to  the  results  discussed  above  (3), 

this  would  result  in  improved  performance.  This  logic  suggests  that  | 

an  analysis  of  the  information  processing  required  of  the  OOD  per  j 

contact  may  permit  direct  prediction  of  OOD  ship  handling  performance. 
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4.2 


Concepts  Suggested  by  the  Research 


Prediction  of  Performance  From  Display  Analysis 
Analysis  IV  provides  identification  of  information  processing 
provided  by  the  OLD  and  PACS  displays.  The  PACS  display  automates 
two  types  of  tasks:  "detect  CPA  violations  and  determine  time  to 

violation"  and  "identify  alternative  course  and  speeds  to  avoid  CPA 
violations."  In  contrast,  the  OLD  display  does  not  automate  those 
tasks  and  thus  they  must  be  performed  by  the  OOD. 

As  stated  previously,  an  analysis  of  the  processing  required 
of  the  OOD  per  contact  for  a given  display  design  may  permit  direct 
prediction  of  total  system  performance.  In  order  to  construct  such 
a prediction  methodology,  assume  that  the  processing  (WL)  workload 
equals  the  sum  of  the  efforts  required  to  process  information  for  each 
component  sub-task  - an  initial  assumption  that  no  parallel  processing 
is  used.  Thus,  with  this  assumption  WL  can  be  expressed  as 

WL  = s T + ? t.  N 

it  J J 

where  T.  represents  the  effort  required  for  information 
processing  sub-task  (i)  that  is  not  a function  of  the 
number  of  contacts,  tj  represents  the  effort  required 
for  information  processing  sub-task  (j)  for  each  contact, 
and  N is  the  number  of  contacts. 
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It  is  also  assumed,  in  order  to  form  the  above  expression,  that  effort 
required  to  process  information  for  each  contact  times  the  number  of 
contacts  (N)  can  be  represented  by,  or  approximated  by,  the  linear 
term  (lN).  Rearranging  the  expression  and  solving  for  N yields: 

WL  - ? T. 

N = 

St. 

J J 

This  is  a general  expression  which  gives  the  number  of  contacts  (N) 
processed  by  the  OOD  as  a function  of  the  information  processing  sub- 
tasks that  must  be  performed  by  the  OOD. 

Numeral  values  for  WL,  T , and  t.  are  required  to 
compute  N.  One  way  to  determine  the  numeral  values  is  to  conduct 
performance  tests  with  several  types  of  displays  and  determine  the 
apparent  N for  each  display.  If  a display  provides  automatic  calculation 
of  one  or  more  sub-tasks,  the  representation  of  the  effort  for  those 
sub-tasks  is  set  to  zero  (i.e.,  T.  = 0,  or  t.  = 0). 

i j 

The  theory  of  equations  says  that  if  there  are  K variables 
then  K equations  are  required  to  solve  for  each  variable.  Thus,  with 
this  method,  the  number  of  different  types  of  displays  tested  must 
be  equal  to  the  number  of  sub-tasks  plus  one.  The  additional  test 
permits  solving  for  the  WL  factor. 


f 
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As  an  example,  consider  the  results  of  Analysis  IV  (shown 
in  Table  10).  According  to  the  analysis,  the  OOD  must  provide  all  of 
the  information  processing  tasks  identified  when  using  the  OLD  display. 

In  contrast,  the  PACS  display  provides  virtually  all  processing  required 
to  perform  Tasks  2 and  3:  "detection  of  potential  CPA  violations,"  and 

"identification  of  candidate  alternative  courses  and  speeds  to  avoid  the 
CPA  violation."  Thus,  the  information  processing  workload  with  the 
OLD  display  can  be  expressed  by: 

WL  = T1  + t (NO)  + t2  (NO)  + tg  (NO)  + Tg 

where 

NO  is  the  number  of  contacts  in  the  purview  associated  with 
the  OLD  display.  The  terms  T , T , t , t , and  t are  defined  in 

1 i 1 c O 

Table  10.  Thus: 


NO  = 


WL  - (T1  + T2) 

<*1  + + V 


Likewise,  the  information  processing  workload  with  the 
PACS  display  can  be  expressed  by: 


WL  = T1  + t3  (NP)  + T2 

where  NP  is  the  number  of  contacts  in  the  purview  associated  with  the 
PACS  display.  Thus: 
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WL  - (T  + T ) 


The  absence  of  tasks  and  t in  the  PACS  workload  expression 
reflects  the  PACS  display  feature  that  provides  virtually  all  the 
computation  required  to  perform  those  tasks. 

Representation  of  Human  Performance  by  the  QMAC  Optimized 
Based  on  the  overall  results  obtained  with  the  OMAC,  it  is 
concluded  that  the  criteria  modeling  approach  offers  a practical  way  to 
model  human  performance  in  an  operational  or  near  operational  problem 
setting.  The  method  seems  to  be  applicable  to  any  manned  system 
problem. 

With  this  modeling  method,  the  subject's  output  at  each 
instant  of  time  is  represented  by  taking  into  account  both  the  instantaneous 
problem  state  and  future  planned  subject  outputs  as  a result  of  projections 
of  controlled  device  responses.  Projections  of  controlled  device  responses 
are  the  expected  future  responses  of  the  device(s)  or  expected  behavior 
of  disturbing  factors  such  as  the  contacts  of  the  ship  control  problem. 

Representative  OMAC's  reveal  the  target  criteria  of  the 
human  controller  and  permit  evaluation  of  his  ability  to  perform 
according  to  that  criteria.  OMAC's  also  reveal  the  constraints  that 
may  be  selfimposed  or  are  inherent  but  limit  the  subject's  performance 


capability 


OMAC's  can  be  developed  for  subjects  working  with  complex 


' 


and  non-linear  tasks.  The  complexity  of  the  task  or  the  device  being 
controlled  does  not  limit  the  identification  of  the  apparent  OMAC. 
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INTRODUCTION 

This  Appendix  provides  a description  of  the  center  CRT 
display  used  for  the  experiments  at  NSRDC . Figures  A1  , A2  and 
A3  are  sketches  of  the  display  format  for  the  OLD,  RVV,  and  PACS 
display  types  respectively.  Table  A1  identifies  the  features  available 
with  each  display.  A description  of  the  features  follow. 

Explanation  of  Display  Symbols 

1 . Contact  symbol  - 1/8  inch  square  marks  the  position 
of  each  contact 

2.  Heading  marker  - heading  cursor  of  own  ship 
OLD  - solid  line  from  center  to  edge  of  display 
RVV  - dashed  line  from  center  to  edge  of  display 
PACS  — solid  line  from  center  to  length  defined 

by  "leaders  adjust" or  "unit  length"  and 

then  dashed  to  edge  of  display 

2 

3.  Upper  left  corner  - northup  or  headup 

4.  Range  - 3,  6,  12,  24  miles  - lower  left  corner 
'Range  XX' 


"leaders  adjust"  and  "unit  length"  refer  to  controls  which  govern 
the  length  of  the  heading  marker. 

2"northup"  and  "headup"  refer  to  the  orientation  of  the  display. 
With  northup  mode  the  display  is  rotated  so  that  North  is  always 
vertical.  With  "headup"  the  display  is  rotated  so  that  ship  heading 
is  vertical. 


■; 
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5.  Ranger  rings  - 0,  1,  5,  10,  20  - lower  right  corner 
'Range  Rings  XX' 

6.  Tab  symbol  '+'  - the  tab  position  is  controllable 
by  the  subject  to  facilitate  acquiring  or  dropping 
contacts,  and  determine  contact  range  and  bearing. 

7 . Contact  Projections 

TVV  - With  this  display  function,  true  velocity 
vectors  are  displayed  for  all  r -quired  contacts 
provided  "leaders  adjust"  or  "ur..t  length"  is  on 
and  set  at  other  than  zero.  A line  is  presented 
from  the  contact  symbol  with  a length  defined  by 
leaders  adjust  or  unit  length.  An  arrow  on  end 
of  the  line  represents  the  predicted  position  of  the 
contact  at  a specified  future  time.  The  words 
"true  vectors"  appear  upper  right  of  the  center 
display. 

RV V - With  this  display,  relative  velocity  vectors 
replace  the  true  velocity  vectors.  The  words 

1 

"relative  vector"  appear  upper  right  of  the  center 
display. 

T rial  Heading  - When  activated  by  the  subject,  a 
cursor  line  appears  extending  from  the  center  to 


W 
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edge  of  the  display.  The  line  may  be  rotated  under 


1 

D 2 


II 

1 


control  of  the  subject.  The  words  'TRIAL  HDC' 
appear  left  side  of  the  center  display. 

T rial  Speed  - When  activated  by  the  subject,  the 
words  'Trial  Speed'  appear  left  side  of  the  center 
display. 

PACS  - With  this  display  function,  the  center 
display  shows  the  locus  of  points  of  contact 
violations  (passing  with  4,000  yards  from  a 
contact)  for  present  and  alternative  own  ship 
courses . 

8.  CPA  Ring  for  RVV  - Range  ring  at  minimum 
miss  distance. 
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NORTH UP 


RANGE  24 
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OLD  SYMBOLS 


1,  2,  3,  4,  5,  9 


RVV  SYMBOLS 

1,  2,  3,  4,  5,  6 (7A,  B,  C,  D),  8 


PACS  SYMBOLS 

1,  2,  3,  4,  5,  6 (7,  A,  C,  D,  E) 


TABLE  A1  DISPLAY  FEATURES  AVAILABLE  FOR 

EACH  TYPE  OF  DISPLAY 
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